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ABSTRACT 
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Specifically, this report s.-;eks to address the question of whether 
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necessary prerequisites for acquiring expertise in economic problem 
solving. The researchers examined literal transcripts generated from 
the "talk-aloud" protocols of 28 participants responding to three 
economic problems. Each sub-group contained four participants: (1) 
high school students who had taken economics and those who had not 
taken economics; (2) undergraduate economics majors and non-majors; 
(3) graduate students in economics; and (4) Ph.D. economists employed 
in public and private forecasting and academic Ph.D. economists. The 
study employed a causal-comparative design with members of the seven 
sub-sample groups identified by the researcher. Three economics 
problems were employed in the study with several phases of data 
gatl'ering involved in the analysis of data. Extensive charts and 
graphs accompany the text. Contains 41 references. (EH) 
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Introduction 

Economic educators have long emphasized that the teaching and learning of economics should 
result in the increased willingness and ability of students to use the discipline of economics in solving 
problems, boUi personal and social. Given this emphasis, the focus of much research in economic education 
has been on the factors affecHng student understanding and application of economic concepts. However, 
recent research calls into question the adequacy of this approach as a means to explore the relationship 
between educating for economic understanding and the goal of better economic problem solving. 
Researchers have investigated the literature concerning schemata and expert problem solving drawn from 
cognitive psj'chology (Miller and VanFossen, 1994; VanSickle, 1992 and VanSickle and Hoge, 1991), which 
seems to hold promise for linking economic education and economic problem solving. 

A previous study by Miller and VanFossen (1994) has described key differences between expert and 
novice problem solvers in economics. However, explicating the distinctions between novices and experts 
only begins the quest to understand how one acquires expertise in economic problem solving and, further, to 
promote experrise through economic educaHon. A crucial step is to understand the attributes associated 
with the construct of relative expertise. That is, differences in expertise not just between experts and novices, 
but among individuals at various points-other than expert and novice-along the path to expertise. Indeed, 
as the characteristics of relative expertise in economic problem solving become known, it nnay be possible to 
design curriculum in economic educaHon that might belter assist students in moving closer to a more 
sophisticated level of economic problem solving ability. 

This paper reports preliminary research into the nature of relative experHse in economic problem 
solving. Specifically, this report seeks to address the question of whether the presence of economic 
knowledge alone accounts for expertise in economic problem solving or whether both economic knowledge 
and the development and employment of economic problem solving strategies are necessary prerequisites 
for acquiring expertise in economic problem solving. 
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The first section briefly describes, within the context of research on expert and novice problem 
solving, why such research is needed. Subsequent sections present the problem within the context of 
research on expert and novice problem solving and the methods, results, and conclusions of the study. 

Context of the Rosearch an d the Problem 

Rrniinmic F Hnrarion anH Prnhlpm Solving 

Since at least the 1961 Report of the mioml Task Force on Economic Education, economic educators 
have long contended that economic education should develop the ability of students to use economics to 
solve problems, a goal that has been ^iterated many times over the years (Saunders, et al., 1993; Bucldes, 
1991; Miller, 1988; Schug, 1985; and Symmes and Gilliard, 1981). Moreover, at least one important result of 
this enhanced economic problem solving ability should be manifest in more effective and responsible 
citizenship (Miller, 1991 and 1989; and Brenneke, 1992). Indeed, increasing democratic citizens' abilities to 
engage in more effective economic problem solving is a.e raison d'etre of economic education. 

Thus the rationale for economic education posits a chain: that more and better economic education 
will lead to greater student understanding of economics which will lead to better economic problem solving 
which, in turn, results in more responsible citizenship (Miller and VanFossen, 1994). Of the links in the 
chain, the first - the relationship of economic education to greater student understanding of economics - 
has received the most attention, with particular emphasis on studies of the impact of economic instruction 
on scores of students on standardized economics tests (See Walstad and Soper, 1991, for numerous 
examples). Until ve^^ recently, however, research on the relationship of conceptual understanding to 
problem solving, the second link, was largely confined to studies comparing students' and economists' 
opinions on various economic issues. The third link has been left largely unexplored. 

There is, however, reason to question wheU^er competence in economics should be equated with 
understanding of economic concepts alone and, further, whether knowledge of economic concepts is 
sufficient, in and of itself, to lead to greater expertise in economic problem solving. Research in expert and 
novice problem solving and schema theory (Voss, et al., 1984, 1989; Chi and Glaser, 1980; Chi, Glaser and 
Rees, 1982; Glaser and Chi, 1988) suggest the need for a more complete conceptualization of economic 
knowledge (VanSickle, 1992; Miller and VanFossen, 1994). "Indeed the noUon of economic knowledge as 
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highly developed schemata (interconnected cognitive structures) suggests a redefinition of economic 
knowledge as inextricably intertwined in a network that includes the linkages among bits of economic 
knowledge and the specialized procedures for using that knowledge" (Miller and VanFosscn, 1994). 
Expertise and Problem Solving 

In studies in fields as diverse as radiology, physics, and, political science, differences in how experts 
solve problems have been shown to depend upon more than simply knowledge of the content of their 
respective disciplines. To take a specific example, Lesgold, et al. 11981) found that experienced radiologists 
were nearly always more successful in their diagnoses than recent interns, despite nearly equal content 
knowledge. Van Sickle (1992) has noted that experts possess more declarative, procedural, and schematic 
knowledge than non^xperts,^ Of these, only declarative knowledge refers to the extent knowledge of a 
specific body of content or discipline. Voss, et al.,(1989) defined procedural knowledge is "the knowledge 
how to/' or perhaps more accurately, as the application of specific procedures or strategies associated with 
the use of declarative knowledge. 

Miller and VanFossen (1994) summarized ten attributes of expert problem solvers drawn from a 
lengthy content analysis of the relevant research literature in cognitive psychology and found evidence that 
these attributes also exist in the domain of economics and were demonstrated by expert econonaic problem 
solvers. These attributes are listed at Figure 1. 

A brief examination of the attributes listed at Figure 1 reveals that more than simply content or 
domain-specific knowledge alone is present in expert problem solvers. Indeed, many of the attributes 
noted in Figure 1 deal specifically with problem solving procedures or strategies that experts employ in 
the process of solving problems (e.g., attributes 1,2,3, and 7). This relarionship-between content 
knowledge and problem solving strategies~has been an area of investigation for a number of studiesin 
this area. 



^ Miller and VanFossen have pointed out that the term "scheiTiatic knowledge" is potentially confusing 
since it implies that it is a component of domain specific knowledge that is somehow separate from 
declarative and procedural knowledge. They argue that it is perhaps better to conceive of this as more 
highly developed domain specific schemata, the integration of declarative and procedural knowledge. 



ERLC 



A 



1. Experts excel mainly in their domain. 

2. Experts perceive relevant patterns in their domains, ll^ese meaningful patterns assist in the 
application of dornain-specific knowledge. 

3. Experts sec and represent problems at a deeper, more principled level than do novices. 

4. Experts spend more time on problem representation. Experts employ a 'work forward' 
strategy that requires greater time allocation for problem identification before the 
application of theory or knowledge. 

5. Experts have strong self-monitoring and self-evaluation skills. 

6. Experts demonstrate more flexibility in the process of problem solving. 

7. Experts identify models as tools in problem solving rather than real-life processes. 

8. Experts possess more domain-specific, declarative knowledge, 

9. Experts have extensive procedural knowledge. 

10. Experts have more highly developed specialized schemata than novices. 
Figure 1. Attributes of Expert Problem Solvers (Miller and VanFossen, 1994) 



For example, Chase and Simon (1973) concluded that while content knowledge competence was 
obviously a prerequisite for acquiring expertise, more than superior content knowledge alone was 
present in expert problem solvers. Simon and Simon (1978) and Chi, et al., (1982) found that the use of 
problem representation strategies were a crucial component of expert problem solving in physics. In 
these studies, novice physicists tended to apply equations quickly and with little discrimination while 
experts concentrated primarily on understanding and categorizing a physics problem before applying 
relevant equations. Further, expert physicists employed a "work-forward" strategy using 
straightforward inferences that focused attention explicitly on a particular solurion (Simon and Simon, 
1978). Novices, on the other hand, employed a "work backward" strategy that forced them to follow a 
limiting set of criteria (a sort of 'checklist* of equations), any of which might have been useful in solving 
the particular problem in question (Simon and Simon, 1978; see also Chi, Glaser and Rccs, 1982). 'Hie 
overarching consensus of these shJdies-and others«-was that both domain-specific content knowledge 
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and the application of specific problem solving processes were important factors in expert problem 
solving. 

Expertise and Economic Education 

That there should be evidence that economic problem solving expertise depends on more than just 
knowledge of economic concepts has been implicit in much of the activity of economic educators over the 
years. For instance. Buckles (1991) has written: "Our teaching of economics can be most effective if we focus 
on the ideas that economics is a 'method' an 'apparatus' and a 'technique'" (p. 24). This statement implicitly 
assumes the existence something like procedural knowledge or specific problem solving strategies; an 
economic reasoning, if you will. Much of the writing about the need or rationale for economic education has 
discussed economics in similar terms. Moreover, some of the materials developed by economic educators 
distinctly emphasizes employing the "method" of economic reasoning, including, for example, Mini-Society 
(Kourilsky, 1983), Capstone: The Nation's High School Economics Course (Reinke, et aL, 1989) and United States 
History: Eyes on the Economy (Schug, et al, 1993). 

If this has been the case, why has so little research been devoted to the acquisition of expertise in 
employing the "method" of (Economics or the relationship between economic concept knowledge and the 
application of that knowledge via an economic reasoning? Also, if the goal of economic education is more 
effective economic problem solving, it is important to know how students become more effective problem 
solvers and what instructional strategies and curriculum materials produce greater expertise. 'Toward this 
end, it is crucial to know how well the patterns of economic reasoning conform to those of economic experts. 
Economic educators might then be better able to design curriculum and instruction that addresses the 
development of economic reasoning based on a firm research foundation" (Miller and VanFossen, 1994). 

This report is intended to be a first effort in determining the relationship between the development 
of economic knowledge, the application of economic reasoning (or the use of economic problem solving 
strategies) and the acquisition of expertise in economic problem solving. 
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Models cf the Nature and Construct nf Relative Economic Expertise 

In the absence of research, one might conceive of any number of possible models that depict the 
construct of expertise in economic problem solving, each wuh attendant implications for instruction and 
curriculum design (See Appendix A). For instance, one 'possibility is a simple continuum. As students 
acquire more domain-specific declarative knowledge, they progress in" regular increments toward greater 
expertise in economic problem solving. This model seems unlikely given the expert novice research outlined 
above and research on schema theory (See Torney-Purta, 1991, for a summary of this line of study) but this 
representation corresponds best with the much of the existing research in economic education. It implies 
that material designers and teachers can concentrate on teaching economic concepts and content, and that 
we can assess expertise by measuring how well students have grasped econonuc concepts. 

A more complicated construct is that greater instruction in economics has little noticeable impact on 
expertise in problem solving until one or more threshold levels are reached. One might think of this as a 
"discontinuous continuum," wherein a student moves ahead on the declarative knowledge dimension of 
expertise without making much discernible progress in problem solving until reaching a tlireshold, where a 
"leap" in problem solving expertise occurs. That such a leap is plausible is suggested by the knowledge 
integration aspect of schema theory (VanSickle, 1992; Tomey-Purta, 1991; VanSickle and Hoge, 1991) . The 
student suddenly "gets it" as disparate economic ideas and strategies for economic reasoning become 
incorporated into her or his schemata. This construct would imply the need to study how this level of 
economic knowledge and reasoning integration takes place and what instruction and materials best promote 
it. 

Another possibility is suggestea by the attributes of experrise as drawn from existing cognitive 
research (Glaser and Chi, 1988; Chi, Glaser and Rees, 1982). In this model, students proceed-probably at 
different rates-down multiple dimensions of the attributes of expertise in problem solving. The third model 
in Appendix A shows one such representation that includes dimensions for problem representation and 
procedural and declarative knowledge-three of the ten attributes listed by Miller and VanFossen (1994). 
This model suggests that researchers should concentrate on understanding how each of the attributes is 
acquired with an eye toward instruction that promotes their acquisition. 
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Perhaps the path to expertise is curvilinear. As depicted in Appendix A, there might be a series of 
levels of expertise with some distribution of; for example, standardized test scores at each level. Greater 
expertise in problem solving might at first respond slowly to successive increments of declarative content 
knowledge (e.g., knowledge of economic concepts). Later, the path of expertise turns more sharply upward 
as students build more highly developed economic schema that integrates elements of economic reasoning 
(the "method" of economics). While the depiction in Appendix A shows this possibiHty as a curve concave 
to the X-axis, a curve convex to the y-axis is also possible. Tl^e latter implies rapid progress in the acquisition 
of expertise with diminishing returns to further experience setting in at some point. The former implies that 
there might need to be substantial conceptual understanding before there is much noticeable improvement 
in economic problem solving. Perhaps researchers would then focus on techniques to change the shape of 
the curve. 

Statement of the Problem 

Obviously, there are other possible models. However, while we know something of the differences 
between expert and novice problem solving in economics (VanPosscn and Miller, 1994; Miller and 
VanPossen, 1994; Son and VanSickle, 1993; VanSickle, 1992), we know almost nothing of how such expertise 
is acquired and the relationship of economic education to the acquisition of expertise. This study is intended 
as a first step in describing the relationship among level of economic knowledge, use of economic problem 
solving strategies and the acquisition of expertise in econonuc problem solving. 

Method 

The researchers examined literal transcripts generated from the 'talk-aloud' protocols of twenty- 
eight participants responding to three economic problen^s. Eacl\ of the following sub-groups contained four 
participants: high school students who had not taken economics (HS NOECON), high school students who 
had (HS ECON), undergraduate non-majors (UNO NOECON), undergraduate economic majors (UND 
ECON), graduate students in economics (GRAD), Ph.D. economists employed in public and private 
forecasting (PHD-FIELD) and academic Ph.D. economists (PHD-UNIVERSITY). 
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Research Design 

The study employed a causal -comparative design. According to Fraenkel and Wallen (1990), causal- 
comparative designs attempt "to determine the cause or consequences of differences that already exist 
between groups or among individuals" (p. 305) and also "involve comparing known groups who have had 
different experiences to deterinine possi >le causes or consequences of group membership" (p. 15). Causal- 
comparative research employs the staHc group comparison design (Campbell and Stanley, 1963). For this 
study, the group difference variable was defined as expected level of relative expertise in economic problem 
solving. This variable was defined as the level of formal economic education attained by a participant (the 
variable PARTLEV). Figure 2 provides a schematic that represents the static group comparison research 
design used in the current study. 



Group Independent Variables 

I 13 relevant indicators of expertise, 

(group characteristics) 



(Groups U-VD 



VII 13 relevant indicators of expertise, 

(group characteristics) 



Figure 2. Design of the Current Study 



Dependent Variable 

relative expertise in economic problem solving 



relative expertise in economic problem solving 



Tlu^ static group comparison design posed several sigiiificant problems with respect quesHons of 
internal and external validity. Among these were: (1) lack of randomization, (2) inability to manipulate 
independent variables, (3) subject selection and (4) data collector bias (McCracken, 1991; Campbell and 
Stanley, 1963). 

The current study addressed the lack of randomization and subject selection issues by creating 
homogenous sub^s^imple grouping and by employing puqiosive Mmpling techniques. Fraenkel and Wallen 
(1990) noted that one way to control extraneous variables, outside of random selection, was to "find, or 
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restrict, one's comparison to groups that are homogenous on that variable" (p. 310). Therefore, the current 
stiidy used an across group comparison based upon participants' expected level of expertise in economic 
problem solving as indicated by the level of formal economic education attained (PARTLEV). Tlius, sub- 
sample group members were relatively equivalent in level of formal economic education and economic 
experience attained. As it was not a goal of this study to engage in an experimental design, the issue of 
manipulating an independent variable was unimportant. 

The quesHon of data collector bias in the analysis of transcribed responses was addressed by having 
two raters independently examine and code a random sample of the respondents' protocols. The author was 
the first rater. The second rater was extremely familiar with this line of study and was -ained using the 
coding rubrics employed by the author. The results of the second rater's coding indicated a very high degree 
of correlation between the researcher's original coding and that of the second rater (correlation coefficient = 
.8341; p < .01). These results may be interpreted as partial validation of the coding process employed by the 
researcher in this study. 
Selection of Participants 

Members of the seven sub-sample groups were idendfied, either directly, or indirectly, by the 
researcher. For example, the academic Ph.D. economists (PHD-UNIVERSITY) were approached based on 
suggestions from a member of the economics faculty at The Ohio State University (OSU) and were all 
members of the OSU economics faculty. 

A similar process was undertaken to secure the pardcipadon of the non-university Ph.D.'s (PHD- 
FIELD). A membership list of the Columbus Assodation of Business Economists (CABE) was used to 
identify participants. T\\e four economists who parHcipated were employed in either public or private 
economic forecasting or analysis. 

A list of second- and third-year OSU graduate students in economics (GRAD) was used to generate 
a mailing to which four OSU graduate students (three second-year and one third-year) responded and 
subsequently participated in the study. A list of undergraduate economics majors at OSU was used to 
produce a mailing, out of which four participants were identified. Two of the participants were of junior 
standing, one was a sophomore, and one was of senior standing. 

o i i 
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The undergraduate, non-economic major participants were members of a social studies education 
methods course taught by the researcher. Three of the four participants had never taken a formal economics 
course (high school nor university). The fourth participant had taken, as part of a Masters' Degree program 
in social studies education, a course titled Curriculum and Instruction in Global Economics, but no other 
formal economics courses. 

The four high school economics students were members of a senior-level econonucs class that the 
primary researcher observed during winter quarter, 1992. The four were selected by the researcher, in 
conjunction with the student's classroom teacher, on the basis of their performance throughout the one- 
semester course. The students had just -completed the economics course when the data were collected. It 
sliould be noted that this course was taught by a veteran high school teacher who had attended several 
workshops on teaching high school econonucs and whose course was based upon the National Council on 
Economic Education's Framework for Teaching the Basic Concepts (Saunders, et al, 1993), considered by many 
to be the major pre-collegiate economics curriculum in use today. 

This high school economics teacher also selected the four student participants who had not taken 
high school economics. These four were also seniors and data on this sub-sample group was also collected 
at the end of the term. Thus, in a broad sense, the two high school student sub-groups were relatively 
equivalent except in formal economic education and economic experience. 
Data Collection 

Data collected and analyzed for the current study took the form of transcribed participant responses 
to three pre-determined economic problems. These responses, or protocols, were audio-taped and then 
literally transcribed by the primary researcher. This data collection stratcg)' is well supported by much of 
the previous expert problem solving research in cognitive psychology (See, for example, Chi and Glaser, 
1980; Glaser and Chi, 1982; and Lesgold, et al., 1981 and Voss, et al., 1983, 1989). The so-called 'talk-aloud' 
strategy used in the shJdy asked participants, to the best of their ability, to verbally express their thought 
processes during the problem solving activity. Tliat is, as participants dealt with various domain-specific 
problems, they were encouraged to "put into words" the processes they engaged in while addressing these 
problems. 
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Participants were given a set of standardized instructions that allowed one minute for the 
respondent to familiarize themselves with each problem. The instructions explicitly encouraged 
respondents to draw any diagrams or graphs that might help them in dealing with the problem in question. 
No pre-set time limit was suggested by the instructions. Rather, the participants were told that they should 
begin when ready and continue speaking until they had, to their own satisfaction, dealt sufficiently with the 
problem. Participants were told to continue analyzing the problem until they felt certain they had exhausted 
their input and felt comfortable with their response. This process was repeated for each subsequent 
problem. 

Three economic problems-based upon those constructed by Miller and VanFossen (1994)-were 
employed in the study. Each of these represented one of three broad areas of economic theory: 
microeconomics, macroeconomics and international trade. As Miller and VanFossen noted, these problen\s 
"were crafted to allow the researchers to detect important differences in economic problem solving that 
might be specific to the individual problems" (p. 15). Further, Miller and VanFossen believed, some issues 
of expertise in problem solving may be domain-specific or idiosyncratic. Thus, it was essential to develop a 
series of economic problems that would generate the broadest range of responses and therefore demonstrate 
the widest range of expertise with respect to economic problem solving. The problenas developed were: 

1. Suppose Congress were to double the current minimum wage of $4.25 an hour to $8.50 an hour. 
Analyze the economic impact of this policy and discuss whether you believe such a policy 
would be a good idea or not and wh^- (microeconomic focus). 

2. In 1929, the so-called Great Depression began. Discuss what you believe caused the Great 
Depression and what, if anything, the federal government should have done to keep economic 
conditions from deteriorating so badly (maaoeconomic focus). 

3. Trade among nations is a perennial economic issue. Suppose that you are the recently appointed 
Secretary of Commerce, and assume that our trade deficit has been growing (the US has been 
buying more goods and services from foreign countries than they have been buying from the 
US). As the Secretary of Commerce, your problem is to design and defend the new 
administration's trade policy. How will you respond (international trade focus)? 

The first phase of analysis involved the coding of protocols for eight variables: absolute number of 
relevant statements (STATE), percentage of relevant statements (RELPER), number of economic concepts 
(CONCEPTS), number of economic models (MODELS), number of concept maps (CONMAPS) and export 

ERLC 



Relative Expertise in Economic Problem Solving 12 

ratio profiles (ERP) for causal, (CAUSERP) prepositional (PROPERP) and problem representation staten^ents 
PROBERP). 

Tliese eight variables, along with the five Pitt Problem Solving Coding System variables outlined 
below, constituted a set of indicators of expertise in economic problem solving. These indicators were 
developed through a content analysis of studies in expert-novice problem solving (See, for example, Voss, 
1989; Glaser and Chi, 1988 and Glaser, 1987) and from a pilot study conducted to develop a model for 
assessing expertise in economic problem solving (Miller and VanFossen, 1994). 

The number of relevant statements (STATE) made by a participant during a response was counted. 
For the current study, a relevant statement was defined as one complete sentence in a respondent's protocol 
that contained relevant economic information or that specifically addressed the problem under 
consideration. Similarly, the number of non-relevant statements were also counted. A ratio of relevant-to- 
non-relevant statements was then calculated (RELPER). 

The number of economic concepts (CONCEPTS) used by a respondent within a protocol were 
counted. For the current study, an economic concept was defined as a class of economic phenomenon that 
possessed common characteristics and /or attributes and that also noted linkages to other, broader more 
inclusive economic concepts. Moreover, for the sake of the current study, economic concepts were deemed 
those concepts whose relation to economic theory was generally accepted, or were considered low-inference 
concepts. 

Data were collected regarding the total number of economic models (MODELS) used in a 
participants protocol. The primary researcher coded and counted the number of times a participant used 
various economic models during a response. During this data collection, an economic model was 
considered to be a complex series of conceptual connections, assumptions and rules for rendering economic 
interaction within a specific area of the domain of economics. Examples of such models include: general 
supply and demand models, aggregate demand and aggregate supply models, exchange rate markets, 
public choice models and industrial organization (I-O) models. 

Data on the use of economic concept maps (CONMAPS) were collected by first identifying the 
economic concepts used in a protocol by a respondent. A conceptual network was defined as an 
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interrelation among economic concepts such that the invoking of one economic concept led to tue invoking 

of one or more other economic concepts. In addition, it was assumed that the use of a concept map by the 

respondent was analogous to the invoking of a specific cognitive structure-much like schemata. Therefore, 

the concept maps took on different levels of complexity as they were associated with the explication of a 

particular economic model. For example, a discussion of the concept of supply and demand required a 

discussion of price and therefore equilibrium price, and so forth. However, this was a general market model. 

A second concept map nught involve linking such a general model with a specific example-a labor market- 

and therefore price with real wage rates. 

Given this, the researchers coded an econonrdc concept map as the following: a low-inference 

connection between two or more economic concepts situated somewhat contiguously within the context of a 

protocol and used in the explication of an economic model. Stated another way, if the resf>ondent connected 

(or invoked) one or more economic concepts shortly after invoking a prior concept, and within the 

framework of an economic model, and this connection was essentially correct economically, the conceptual 

string created by the respondent was coded as a concept map. The following excerpt from a respondent's 

protocol is an example of such a concept map: 

"So this is short run, very short run, it is going to happen overnight. The there will a gap in quantity of 
labor demanded and quantity of labor supplied with unemployment resulting.," 

For this respondent, the use of the concept short run was followed closely by the invoking of three 
other, related concepts. These concepts were related in the sense tliat, economically, the concept of short run 
implies no time for markets to adjust to changes in factor inputs or other shocks and, therefore, the issue of 
"gaps in quantity of labor demanded,,." occur only in the short run. Moreover, this map is directly related to 
the application of a generalized market model. This case is a more sophisticated example, as the issue of 
time, and its relationship to the market, is implied. Furthermore, this concept map is directly related to the 
first concept employed by the respondent: short run. 

Data concerning respondent's problem representation statement, causal statement and propositional 
statement Expert Ratio Profiles (ERPs) were coded using criteria previously employed by Miller and 
VanFossen (1994). Statements were classified as problem representation (PROBERP) if they demonstrated 
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an attempt to re-order or re-construct a problem in an effort to discover connections, or to sort the problem 

into more accessible algoritl\nns. Further, problem representation statements demonstrated some level of 

planning on the part of a respondent with respect to the problem and processes used to address the problem. 

ITie follov/ing excerpt from a respondent's protocol illustrates a problem representation statement: 

"...if you're living in a world of fixed exchange rates, that sets up one set of problems. If you're living 
in a world with flexible exchange rates that sets up a different set of problems." 

Similarly, statements w^ere coded as propositional (PROPERP) If they contained 
"links.. .which.. .resembled the 'if part of an 'if.. .then' statement" (Miller and VanFossen, 1994, p. 17). Thus, 
statements that represented some level of logical connection between an economic condition and an 
economic outcome were coded as propositional. The example used above to illustrate problem 
representation is also an example of a propositional statement. A second example of such a prepositional 
statement is "if you double the minimum wage, that means that more people will not be employed." 

Finally, statements that clearly established links of causality (CAUSERP) were coded as causal 
statements. These causal links were defined as statements that made an 'A causes B' distinction. Although 
similar in nature to the propositional statement, the causal statement involved the demonstration of a more 
direct economic connection rather than a hypothesis for examination. For example, a respondent stated that 
the income tax increase of 1932 led to a fall in disposable income and therefore a drop in aggregate demand. 
One can classify such a statement as fitting the 'A causes B' model. 

The second phase of data gathering undertaken by the researcher involved coding the transcribed 
protocols in terms of the Pitt Problem Solving Coding System (PPSCS). First developed by Pitt (1983), the 
PPSCS (See Figure 2) coded qualitative data, such as the current study's transcripts, into one of six categories 
of strategies used in problem solving (general problem solving, feedback, pattern extraction, hypothetico- 
deductive, evaluation and heuristics) by integrating constant comparison analysis (Glaser and Strauss, 1967). 
Pitt (1983) argued that the coding system developed in her study provided a "comprehensive, empirical 
instrument to code heterogeneous verbal protocols in terins of tlie types of processing function each verbal 
proposition represents" (p. 551). 

ERLC ^ 
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Pitt (1983) provided operational definitions for each of the six coding classifications in the PPSCS. 
For the purposes of the current study, general problem solving strategies included defining an initial state ot 
the problem, defining the goals involved, and being able to identify necessary data needed in solving a 



General IVoblem Solving 
SRIO. Define initial state 
SRll. Define goal state 
SR12. Identify data needed 

Feedback 

SR17. Identify feedback 
SRI 8 Tag new information 
SR19 Organize new information 

Pattern Extraction 
SR2Z Extract patterns from data 
SR23. Summarize relevant patterns 

HypotheticO'deductive 
SR 7. Formulate hypjothesis 
SR 8. Define predictions 
SR20. Match data to predictions 
SR21. Determine trutn values of predictions 



Evaluation 

SR 2. List assumptions 

SR 4. Select evaluative criteria 

SR 5. Assign priorities 

SR 6. List relevant information 

SR15. Edit algorithm 

Basic Heuristics 
SR 1. List given information 
SR 3. List questions 
SR 9. Select questions 
SR13. Identify set of available algorithms 
SRI 4. Select algorithms 
SR16. Execute algorithm 
SR24. Output conclusions 



Figure 3. Components of the PPSCS Model: Problem Solving Strategies and Subroutines (SRKPitt, 1983) 



problem. Feedback strategies identified and incoiporated new information as it became available during the 
problem solving process. Pattern extraction strategies referred to the identification of relevant patterns, 
svrnmetries or regularities in the assembled data. Hx/pothetico-deductive strategies involved the formulation 
of hypotheses, engaging in predictions and analyzing the validity of these predictions. Evaluation strategies 
suggested that problem solvers select evaluative criteria, assign priorities, and revise the problem solving 
strategy based upon tl^e evaluation. Finally, Pitt identified a sixtli category: basic heuristics. Basic heuristic 
strategies represented an abbreviated heuristic that can suffice for simple, familiar problems. This 
classification of strategies is more complex than simple trial-and error, yet is loo simplistic for more complex 
problems. For purposes of the current study, however, data were coded on only the first five categories as 
the researchers believed that the basic heuristics category generated data that was very similar to the ERP 
data already calculated for each participant. 
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Using the coding classificalions outlined briefly above, and the constant comparison analysis 
technique, the researcher coded each protocol using the PPSCS. Mean levels for each PPSCS variable were 
then calculated for each of the seven sub-sample groups. 
Data Analysis 

Data on the variables identified in the current study (the indicators of relative expertise in 
economic problem solving) were examined using factor analytic techniques. These statistical techniques 
have been described as a method "to discover or construct from a larger group of observed 
characteristics, or items, a small set of general characteristics, or factors, various combinations of which 
will produce each of the observed patterns of iten\s" (Selvin, 1972, p 255; italics original). Factor analysis 
may also be defined as the use of "a variety of statistical techniques whose common objective is to 
represent a set of variables in terms of smaller number of hypothetical variables" (Kim and Mueller, 
1978, p. 9). However, the fundamental assumption in factor analysis is that the first, larger set of 
variables possess some level of association, as measured by a correlation analysis. Factor analysis is 
therefore, as Thorndike (1982) noted, u-:>ed "to find an underlying structure" and to "identify a small 
number of fundamental trait dimensions" (p. 277). 

For example, in the current study, the various indicators of expertise investigated (e.g., number 
of relevant statements, number of econonrdc concepts, etc.) represented a set of variables (items) that may 
act to define a hypothetical construct known as "expertise in economic problem solving." Thorndike 
(1982) noted that, "a factor is a new variable generated by a linear combination of the original (items)" 
and that "the hope is tliat judicious development of the factors can produce variables that imply clear 
and meaningful.. .constructs" (p. 277-279). Factor analysis was employed in this study to determine if 
any related factors might better define the broad construct "expertise in economic problem solving." 

Two categories of factor analysis exist: exploratory and confirmatory. The fundamental 
difference between exploratory and confirmatory factor analysis concerns whether or not hypotheses are 
tested. Exploratory factor analysis "may be used as an expedient way of ascertaining the minimum 
number of hypothetical factors that can account for observed covariation, and as a means for exploring 
data for possible data reduction" (Kim and Mueller, 1978, p. 9). Confirmatory factor analysis, on the 
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other hand, "is used as a means of confirming a certain hypothesis" (Kim and Mueller, 1978, p. 9). This 
hypothesis is likely to regard the number of factors involved and which variables are most closely 
associated with which factor. The current study employed both exploratory and confirmatory factor 
analysis. 
Factor Analysis 

The first step in this factor analysis was to generate a correlation matrix for the variables under 
consideration. This was done in order to determine tl^e degree of intercorrelation that existed among the 
thirteen variables. It was apparent from the results presented in the correlation matrix (see Appendix B) 
that each of the thirteen variables was moderately to highly correlated with the twelve remaining 
variables. Tliis high degree of intercorrelation among the variables implied that the application of factor 
analytic techniques was indeed warranted (Kim and Mueller, 1978). 

Next, an exploratory factor analysis was conducted using a Kaiser normalization and an 
unweighted least squares solution. The results (Table 1) indicated that two factors satisfied the 
minimum Eigenvalue requirement (Eigenvalue > 1.0). 



Table 1 

Initial Exploratory Factor Analysis: Final Statistics 



VARIABLE 



COMMONALITY 



FACTOR EIGENVALUE PCT OF VAR CUM PCT 



PITTFEED 

PITTEVAL 

PITT PAT 

CONMAPS 

MODELS 

PROBER? 

PROPERP 

RELPER 

STATE 

CONCEPTS 

PITTHD 

PITTGPS 

CAUSERP 



.78605 
.73269 
.72610 
.90772 
.84366 
.60725 
.35475 
.61177 
•92705 
.92158 
.77564 
.83007 
.68514 



8.39890 
1*31058 



64.6 
10.1 



64.6 
74.7 



OBLIMIN ROTATION KAISER NORMALIZATION OBLIMIN CONVERGED IN 12 
ITERATIONS . 
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Using the results of Uiis exploratory factor analysis, a second, confirmatory factor analysis was 
conducted. Again, this analysis employed an unweighted least squares solution. In addition, a factor 
rotation was conducted in order to present a simpkT, more easily interpreted solution. Because the 
factor correlation matrix (See Table 2) suggested the two factors were correlated, an oblique rotation 
technique (using the SPSS Oblimin command) was employed. 

The effect of the oblique rotation was to separate the factor loadings (easily interpreted as a 
coefficient representing the level of intercorrelation among the two identified factors and each of the 
thirteen variables in question) onto these two factors. The variables that loaded appreciably (> .70) onto 
the first factor included number of economic statements used (STATE), number of economic concepts 
used (CONCEPTS), number of economic models used (MODELS), number of concept maps used 
(CONMAPS) and Pitt general problem solving statements used(PITTGPS). 

This first factor was designated as Economic Knowledge and Knowledge Structures. This 
desigiiation was based on the operational definitions of the variables that loaded onto this factor. 
Indeed, four of the five variables that loaded very heavily (> .90) onto the first factor (STATE, 
CONCEPTS, MODELS, CONMAPS) were treated as indicators of economic knowledge by the current 
study. The extremely high factor loadings for these four variables may be interpreted as evidence of a 
very high level of inter-correlation between these four variables and this first factor-Economic 
Knowledge and Knowledge Structures. 



Table 2 

Factor Correlation Matrix: Factor 1 with Factor 2 

FACTOR 1 FACTOR 2 



FACTOR 1 
FACTOR 2 



.60527 
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Table 3 

Factor Structure Matrix: Loadings for All Variables on Factor 1 and Factor 2 



Variable FACTOR 1 FACTOR 2 



CONCEPTS 


.95984 




STATE 


.95534 




CONMAPS 


.94984 




MODELS 


.91815 




PITTGPS 


.88255 




PITTPAT 




.82903 


CAUSERP 




.82151 


PITTHD 




.81003 


PITTFEED 




.80075 


PITTEVAL 




.78955 


RELPER 




.78113 


PROBERP 




.77900 


PROPERP 




.59391 



A strong case can be made for this conclusion, as participant's economic knowledge was most 
certainly a function of the presence of economic concept knowledge (CONCEPTS), a kiiowledge of 
economic models (MODELS) and the linkages between such concepts and models (CONMAPS). 
Further, the number of relevant economic statements (STATE) made was clearly an indicator of the 
strength of a respondent's economic knowledge. Thus, four of the five variables that loaded onto the 
Economic Kiiowledge and Knowledge Structures factor were variables that corresponded well with the 

I 

relative level of economic knowledge demonstrated by participants involved in current study. 

The fiftli variable (PITTGPS) was not used as an indicator of economic knowledge in tlie current 
study. Rather, PITTGPS was taken-as were all the Pitt Problem Solving variables~to be an indicator of 
the level of problem solving strategies employed by participants. Indeed, the PITTGPS measure was 
determined by coding the number of respondent's statements tliat employed general problem solving 
strategies. 

At least one potential explanation exists for this rounter-intuitive n^sult. Indeed, it seems 
plausible tliat certain goteral economic problem solving abilities~-but not necessarily advanced problem 
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solving abilities-arc a direct function of the level of participant's economic knowledge. If tliis is the 
case, then the PITTGPS could be interpreted as a secondary indicator for the level of econonnic 
knowledge demonstrated by participants. 

The results of the confirmatory factor analysis offer some support for this position. It is 
noteworthy that the PITTGPS variable loaded heavily upon both factors [Factor 1 (Economic Knowledge 
and Knowledge Structures) = .8825; Factor 2 (Economic Problem Solving Strategies) = .7143]. These 
loadings demonstrated a high degree of intercorrelation between the PITTGPS variable and both Factor 1 
and Factor 2. 

The variables that loaded appreciably (> .60) onto the second variable included the eight 
remaining variables: problem representation ERP (PROBERP), causal statement ERP (CAUSERP), 
propositional statement ERP (PROPERP), percentage of relevant statements used (RELPER), Pitt pattern 
extraction (PITTPAT), Pitt hypothetico-dcductive (PITTHD), Pitt feedback (PITTFEED) and Pitt 
evaluation (PITTEVAL). All factor loadings and the factor structure matrix are presented at Table 3. 

Based on the operational definitions of seven of these eight variables, this factor was designated 
as Economic Problem Solving Strategies. In fact, the seven (PROBERP, PROPERP, CAUSERP, PITTHD, 
PITTFEED, PITTPAT, PITTEVAL) that loaded appreciably (> .60) on this factor were identified 
specifically for the purpose of analyzing the cognitive processing (Miller and VanFossen, 1994) and/ or 
problem solving abilities (Pitt, 1983) of participants in the current study. These cognitive processes 
(problem representation and the use of propositional and causal statements) and problem solving skills 
(pattern extraction, feedback, hypothetico-deductive statements and evaluation) have been well 
documented as essential attributes of expertise in problem solving (See Pitt, 1983; Chi and Glaser, 1983; 
Losgold,et al., 1981). 

Tlie eighth variable that loaded appreciably (> .60) onto the Economic Problem Solving 
Strategics factor was the percentage of relevant statements used by respondents (HELPER), This result 
seemed out of place given the strong correlation between the factors and the high level of 
correspondence among variable loadings and expectations regarding such loadings. Indeed, it would 
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seem that the HELPER, as a direct function of STATE, sliould have corresponded with the loadings 
associated with STATE. This was clearly not the case. 

However, these counter-intuitive results could bo a function of the fact that the use of a lower 
percentage of relevant statements by participants may reflect both a low level of economic knowledge 
and an inability to reason effectively about a problem due to this lack of knowledge. In this case, the 
loadings may not have been as expected. That is, if participants used a lower ratio of relevant 
statements, this may demonstrate a lack of ability to problem solve effectively due to a lack of some 
necessary iTiiriimum level of economic knowledge. 

It may also be the case that this variable-RELPER-is a less than satisfactory variable in terms of 
representing what has been measured. Thus, this "crossing over" may be a funcrion of a variable 
measuring something altogether different than what was proposed by the researcher. Indeed, this 
variable was developed to correspond with Ennis and Safrit's 11991) use of "incorrect" response 
statements as used in exercise science problem solving, something that could be measured given the 
nature of the problems they presented. 

Ennis and Safrit examined the ratio of correctly reasoned response statements to incorrectly 
reasoned statements. The Ennis and Safrit study, however, used a tightly constrained problem set for 
which broadly accepted agreement-across the domain of exercise sdence-on the nature of "correctness" 
could be gauged Economics, as a domain, however, does not necessarily possess such broad, inter- 
domain agreement in all areas. Thus, as Miller and VanFossen (1994) noted, "the level of theoretical 
agreement among economists varies aaoss miao, macro, and international economics" and therefore "it 
is possible that experts might distinguish among different theoretical approaches in some problems" (p. 
15). 

Given this nature of the domain of economic theor\', the current study attempted to replace 
Ennis and Safrifs notion of "incorrectness" with that of "non relevance." Ai noted above, however, the 
correspondence between "correctness" and "relevance" may be less than assumed for purposes of this 
study and thus the variable RELPER may not bo a reasonable proxy for tliis concept. 
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Discussion 

A factor analysis waF conducted on thirteen variables identified as indicators of expertise in 
economic problem solving (Vc.nFosscn and Miller, 1994; VanFossen, 1994) in order to detern\ine whether 
any underlying factors might be identified. The results indicated that these variables loaded appreciably 
upo. . and thus could be used to define, tw^o factors: Economic Knowledge and Knowledge Structures 
and Economic Problem Solving Strategies. These results also implied that both content knowledge and 
the use of a problem solving process component seemed to play an important role in distinguishing the 
relative levels of expertise that existed among this study's sub-groups. Indeed, these results seemed to 
provide further evidence for the integral relationship between possessing economic knowledge and the 
use of economic problem solving strategies in acquiring expertise in economic problem solving. 

The potential implications of these findings for economic education and economic educators 
seem quite relevant, especially in the realm of curriculum design and development. Although these 
results must be interpreted cautiously due to the small size of this sample, it would appear that success 
in economic problem solving turns not only on a student's knowledge of economic concepts, but also on 
the student's application of economic problem solving strategies or reasoning. Indeed, the experts in 
this study clearly demonstrated the use of both. 

This notion of using economic problem solving btrategies seems to mirror certain current 
curriculum in economic education. Notably, the application of what Buckles (1991) referred to as the 
method of economics is apparent in the National Council on Economic Education's (NCEE) Capstone: The 
hlation^s Economics Course (Reinkc, et al., 1989) curriculum guides. T\\\s integrative economics program 
helps students to develop an economic way of tWnking that relies heavily on so-called "economic 
mysteries" that require students to apply both economic concepts and a problem solving strategy (e.g., the 
Handy Dandy Guide to Solving Economic Mysteries). Recently, the NCEE's Eyes on the Economy: Economics 
inU.S. History (Schug, et al., 1993) has furthered this pedagogical approach-each major section in the 
curriculum is predicated on an intriguing mystery that students must use economic reasoning to solve. 

If economics education is to continue to claim the development and promotion of economic 
literacy among high school students as its primary ain^ then we must continue to apply the knowledge 
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gleaned from such areas as cognitive psychology in order to develop and integrate nnore effective 
economics curriculum along the lines of those noted briefly above. Further study in this area, may 
indeed prove fruitful to our curriculum development efforts. 
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APPENDIX A: 

Hypothetical Constructs of Expertise in Economic Problem Solving 



1 . Continuum 



Novice Expert 
2. Discontinuous Continuum 

/ H 

/ Expert 

I ' 

Novice 



3. Multiple Continua 




4. Curvilinear* 




Novice • The dxted hue m\ this construct cufinccts the m^ans of a rai^gc o( scores, of <x\yer 

indjcatofs, aswcwccd wtth a sub-sample of the sruJv and represents one hypochettcal 
"shape' of expeftae. 
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